flowchart LR X[Predictor\nof interest] -- Causal? --> Y[Outcome] X[Predictor\nof interest] <-- Association --> Z[Confounder] Z[Confounder] -- Causal --> Y[Outcome]
Confouding, Effect Modification, and Mediation
Lecture 06
1 Overview
1.1 Scientific questions
Most often scientific questions are translated into comparing the distribution of some response variable across groups of interest
Groups are defined by the predictor of interest (POI)
Categorical predictors of interest: Treatment or control, knockout or
wild type, ethnic group
Continuous predictors of interest: Age, BMI, cholesterol, blood pressure
Often we need to consider additional variables other than POI because …
We want to make comparisons in different strata
- e.g if we stratify by gender, we may get different answers to our scientific question in men and women
Groups being compared differ in other ways
Confounding: A variable that is related to both the outcome and predictor of interest
Less variability in the response if we control for other variables
Precision: If we restrict to looking within certain strata, may get smaller $\sigma^2$
Covariates other than the Predictor of Interest are included in the model as\…
Effect modifiers
Confounders
Precision variables
Not necessarily mutually exclusive
2 Effect Modification
The association between the Response and the Predictor of Interest differs in strata defined by the effect modifier
Statistical term: “Interaction” between the effect modifier and the POI
2.1 Effect modification depends on the measure of effect that you choose
Choice of summary measure: mean, median, geometric mean, odds, hazard
Choice of comparisons across groups: differences, ratios
2.2 Examples of Effect Modification
2.2.1 Example 1: Is serum LDL by gender modified by smoking?
| Mean | Mean | Median | Median | |
| Women | Men | Women | Men | |
| No Smoke | 120 | 122 | 120 | 115 |
| Smoke | 133 | 122 | 133 | 124 |
| Diff | -13 | 0 | -13 | -9 |
| Ratio | 0.90 | 1 | 0.90 | 0.93 |
Effect modification for mean, not really for median
- Holds for both difference or ratio
2.2.2 Example 2: Creatinine by stroke (modified by gender?)
| Mean | Mean | Median | Median | |
| Women | Men | Women | Men | |
| No Stroke | 0.72 | 1.08 | 0.7 | 1.1 |
| Stroke | 1.01 | 1.51 | 1.0 | 1.5 |
| Diff | -0.29 | -0.43 | -0.3 | -0.4 |
| Ratio | 0.71 | 0.72 | 0.70 | 0.73 |
Effect modification for difference, not really for ratio
- True for Mean or median
2.2.3 Example 3: Stroke by smoking (modified by gender?)
| Proportion | Proportion | Odds | Odds | |
| Women | Men | Women | Men | |
| No Smoke | 0.10 | 0.16 | 0.03 | 0.19 |
| Smoke | 0.16 | 0.26 | 0.19 | 0.35 |
| Diff | -0.06 | -0.10 | -0.16 | -0.16 |
| Ratio | 0.62 | 0.62 | 0.16 | 0.54 |
Proportion: No effect modification for ratio, small amount for difference
Odds: No effect modification for difference, yes for ratio
2.2.4 Example 4: Stroke by smoking (modified by CVD?)
| Proportion | Proportion | Odds | Odds | |
| None | CVD | None | CVD | |
| No Smoke | 0.02 | 0.33 | 0.02 | 0.50 |
| Smoke | 0.04 | 0.50 | 0.04 | 1.00 |
| Diff | -0.02 | -0.17 | -0.02 | -0.50 |
| Ratio | 0.50 | 0.67 | 0.50 | 0.50 |
Effect Modficiation?
Proportion: Yes for ratio, yes for difference
Odds: Yes for difference, no for ratio
2.2.5 Example 5: CHD by current smoking (modified by gender?)
| Proportion | Proportion | Odds | Odds | |
| Women | Men | Women | Men | |
| No Smoke | 0.18 | 0.26 | 0.22 | 0.36 |
| Smoke | 0.05 | 0.24 | 0.05 | 0.32 |
| Diff | 0.13 | 0.02 | 0.17 | 0.03 |
| Ratio | 3.60 | 1.08 | 4.17 | 1.11 |
Effect Modfication?
Proportion: Yes for ratio, yes for difference
Odds: Yes for difference, yes for ratio
2.2.6 Example 6: CHD by ever smoke (modified by gender?)
| Proportion | Proportion | Odds | Odds | |
| Women | Men | Women | Men | |
| Never Smoke | 0.16 | 0.25 | 0.19 | 0.33 |
| Ever Smoke | 0.16 | 0.26 | 0.19 | 0.35 |
| Diff | 0.00 | -0.01 | 0.00 | -0.02 |
| Ratio | 1.00 | 0.96 | 1.00 | 0.95 |
Effect Modfication?
Proportion: No for ratio, no for difference
Odds: No for difference, no for ratio
2.2.7 Summary comments on examples
If there is an effect, will see effect modification on at least one of the difference and ratio scale
If there is no effect (example 6), will see no effect modification on both difference and ratio scale
In real world, will usually see effect modification on both scales. The real question is the effect modification scientifically meaningful.
If we find there is important effect modification, science will go forward estimating effects separately
Models with interaction terms are useful for testing if effect modification is present (statistically)
Aside: Be careful when comparing two ratios
How close are two ratios?
0.20 and 0.25 VERSUS 5.0 and 4.0?
0.10 and 0.15 VERSUS 10.0 and 6.7?
Compare the ratio of ratios, not the difference
We might consider ratios to be more different when both ratios are \(>1\) than when both are \(<1\). But, that would be wrong.
2.3 Analysis of Effect Modification
When the scientific question involves effect modification
Conduct analysis within each stratum separately
If we want to estimate the degree of effect modification or test its existence, use a regression model including
Predictor of interest (main effect)
Effect modifying variable (main effect)
A covariate modeling the interaction (usually a product)
2.3.1 Impact of ignoring effect modification
By design or mistake, we sometimes do not model effect modification
Might perform
Unadjusted analysis: POI only
Adjusted analysis: POI and third variable, but no interaction term
If effect modification exists, an unadjusted analysis will give different results according to the association between the POI and effect modifier in the sample
If the POI and the effect modifier are not associated
Unadjusted analysis tends toward an (approximate) weighted average of the stratum specific effects
With means, exactly a weighted average
With odds and hazards, an approximate weighted average (because they are non-linear functions of the mean)
If the POI and the effect modifier are associated in the sample
- The “average” effect is confounded and thus unreliable (variables can be both effect modifiers and confounders)
If effect modification exists, an analysis adjusting only for the third variable (but no interaction) will tend toward a weighted average of the stratum specific effects
- Hence, an association in one stratum and not the other will make an adjusted analysis look like an association (provide the sample size is large enough)
3 Confounding
3.1 Simpson’s Paradox
Confounding has its roots in Simpson’s Paradox
Given binary variables \(Y\) (response), \(X\) (POI), and \(Z\) (strata) it is possible to have …
\[\textrm{Pr}(Y=1 | X=1, Z=1) > \textrm{Pr}(Y=1 | X=0, Z=1)\] \[\textrm{Pr}(Y=1 | X=1, Z=0) > \textrm{Pr}(Y=1 | X=0, Z=0)\]
… but to also have …
\[\textrm{Pr}(Y=1 | X=1) < \textrm{Pr}(Y=1 | X=0)\]
3.1.1 Example: Probability of death (Y) at two hospitals (X) stratified by poor patient condition (Z)
- Question: Which hospital do you want to be treated at?
- Consider the results overall (averaging over Z) and conditional on Z below
| Overall | Died | Survived | Death Rate |
|---|---|---|---|
| Hospital A | 16 | 784 | 2.0% |
| Hospital B | 63 | 2037 | 3.0% |
| Good Condition | Died | Survived | Death Rate |
|---|---|---|---|
| Hospital A | 8 | 592 | 1.3% |
| Hospital B | 6 | 594 | 1.0% |
| Poor Condition | Died | Survived | Death Rate |
|---|---|---|---|
| Hospital A | 8 | 192 | 4.0% |
| Hospital B | 57 | 1443 | 3.8% |
Ignoring condition, Hospital B has a higher death rather. However, within both poor and good condition, Hospital B has a lower death rate.
- Poor condition is a confounder. Hospital B has more subjects with poor condition and subjects with poor condition have a higher death rate.
3.2 Definition of Confounding
The association between a predictor of interest and the response is confounded by a third variable if
The third variable is associated with the predictor of interest in the sample, AND
The third variable is associated with the response
Causally (in truth)
In groups that are homogeneous with respect to the predictor of interest
Not in the causal pathway of interest
We must consider our belief about the causal relationships among the measured variables
There is no statistical test for causality
Inference about causation comes only from the study design
BUT, consideration of the causal relationships helps us to decide which statistical questions to answer
Classic confounder
- A clear case of confounding occurs when some third variable is a “cause” of both the POI and response
We generally adjust for such a confounder
3.2.1 Directed Acyclyic Graph
- Example: Ice cream (POI), murder rate (outcome), and temperature (confounder) in New York City during the summer
flowchart LR X[Ice Cream] -- Causal? --> Y[Murder Rate] X[Ice Cream] <-- Association --> Z[Air temperature] Z[Air temperature] -- Causal --> Y[Murder Rate]
3.2.2 Causal pathways
A variable in the causal pathway of interest
Not a confounder, so we would not adjust for such a variable
If we did adjust, we would lose ability to detect associations between the POI and the outcome
Example: Second hand smoke (POI), stunted growth (confounder), FEV1 (outcome)
Scientific question is about the impact of smoking on lung function
- Stunted growth addresses lung anatomy, not lung function, which we don’t care about it
flowchart LR X[Second hand smoke] --> Y[FEV1] X[Second hand smoke] --> Z[Stuntend growth] Z[Stunted growth] --> Y[FEV1]
A variable in the causal pathway not of interest
- However, we want to adjust for a variable in a causal pathway that is not of interest
- Example: Work stress causing ulcers by hormonal effects versus alcoholism
- Directed Acyclyic Graph
- We can adjust for alcoholism to estimate the path through horomonal effects
- Alternatively, we can adjust for hormonal effects to estimate the effect through alcoholism
flowchart LR X[Work stress] --> W[Hormonal Effects] X[Work Stress] --> Z[Alcholism] W[Hormonal Effects] --> Y[Ulcers] Z[Alcholism] --> Y[Ulcers]
Surrogate for response
Adjustment for a surrogate is a bad idea
As the name implies, surrogates are a substitute for the response variable
Directed Acyclyic Graph where forced vital capacity (FVC) is a surrogate for forced exp
flowchart LR X[Second hand smoke] --> Z[FVC] Z[FVC] --> Y[FEV1]
Many other (complicated) patterns possible
- Greenland, Pearl, and Robins. Causal Diagrams for Epidemiologic Research. Epidemiology. (1999) http://www.jstor.org/stable/3702180
- https://www.dagitty.net/
3.3 Diagnosing Confounding
Confounding typically produces a difference between unadjusted and adjusted analyses
This symptom is not proof of confounding
Such a difference can occur when there is no confounding
Symptom is more indicative of confounding when modeling means (linear regression) than when modeling odds (logistic regression) or hazards (Cox, proportional hazards regression)
Estimates of association from unadjusted analysis are markedly different from estimates of association from adjusted analysis
Association within each stratum is similar to each other, but different from the association in the combined data
In linear regression, differences between adjusted and unadjusted analyses are diagnostic of confounding
Precision variables tend to change standard errors, but not slope estimates
Effect modification would show differences between adjusted analysis and unadjusted analysis, but would also show different associations in the strata
More difficult to diagnosis confounding with non-linear functions of the mean
Common non-linear functions: Odds (odds ratios), hazards (hazard ratios)
May show the symptoms of confounding when confounding is not present
Adjusting for precision variables can appear to be confounding
In logistic and PH regression, difference between adjusted and unadjusted analyses are more difficult to judge
Comparison in more homogeneous groups (i.e. after adjustment for a precision variable) will drive slope estimates away from the null
Example: Suppose you have a sample where 50% of the subjects die
What is the variability?
We can reduce this variability by changing \(p\), the probability of death
Estimate \(p\) in different stratum. One stratum may have a higher \(p\), another a lower \(p\).
By making the estimate more precise, we have also impacted the mean
4 Precision Variables
4.1 Overview
Sometimes the scientific question to be answered is chosen based on which questions can be answered most precisely
In general, questions can be answered more precisely when the within group distribution is less variable
Comparing groups that are similar with respect to other important risk factors decreases variability
The precision variability is independent of the cause of the response
If we adjust for such a variable, we tend to gain precision
Directed Acyclyic Graph:
flowchart LR X[Predictor] --> Y[Outcome] Z[Precision] --> Y[Outcome]
Standard errors are the key to precision
Greater precision is achieved with smaller standard errors
Standard errors are decreased by either increasing \(V\) or decreasing $n$
Typically: \(se(\hat{\theta}) = \sqrt{\frac{V}{n}}\)
Width of CI: \(2 \times (\textrm{crit value}) \times se(\hat{\theta})\)
Test statistic: \(Z = \frac{\hat{\theta} - \theta_0}{se(\hat{\theta})}\)
Options for increasing precision
Increase sample size
Decrease $V$
(Decrease confidence level)
4.2 Adjusting for Precision Variables
4.2.1 Precision for Difference of Independent Means
Independent observations where group 1 has a different mean and variance than group 2
$ Y_{ij} \sim (\mu_j, \sigma_j^2), j = 1, 2; i = 1, \ldots, n_j$
$n = n_1 + n_2$; $r = n_1 / n_2$
$\theta = \mu_1 - \mu_2$,
$\hat{\theta} = \overline{Y}_1 - \overline{Y}_2$
$V = (r+1)(\frac{\sigma_1^2}{r} + \sigma_2^2)$
$se(\hat{\theta}) = \sqrt{\frac{V}{n}} = \sqrt{\frac{\sigma_1^2}{n_1} + \frac{\sigma_2^2}{n_2}}$
Might control for some variable in order to decrease the within group
variability
Restrict population sampled
Standardize ancillary treatments
Standardize measurement procedure
#### Precision for Linear Regression
Independent continuous outcome associated with covariate ($X$)
$\textrm{ind } Y_i | X_i ~ \sim(\beta_0 + \beta_1 X_i, \sigma^2_{Y|X}), i = 1, \ldots, n$
$\theta = \beta_1, \hat{\theta} = \hat{\beta_1}$ from LS regression
$V = \frac{\sigma^2_{Y|X}}{\textrm{Var}(X)}$
$se(\hat{\theta}) = \sqrt{\frac{\hat{\sigma}^2_{Y|X}}{n \hat{\textrm{Var}}(X)}}$
Adjusting for covariates ($W$) decreases the within group standard
deviation
$\textrm{Var}(Y | X)$ versus $\textrm{Var}(Y | X, W)$
Independent continuous outcome associated with covariate ($X$) and
precision variable ($W$)
$\textrm{ind } Y_i | X_i, W_i ~ \sim(\beta_0 + \beta_1 X_i + \beta_2 W_i, \sigma^2_{Y|X,W}), i = 1, \ldots, n$
$\theta = \beta_1, \hat{\theta} = \hat{\beta_1}$ from LS regression
$V = \frac{\sigma^2_{Y|X,W}}{\textrm{Var}(X)(1-r^2_{X,W})}$
$se(\hat{\theta}) = \sqrt{\frac{\hat{\sigma}^2_{Y|X}}{n \hat{\textrm{Var}}(X)(1-r^2_{X,W})}}$
$\sigma^2_{Y|X,W} = \sigma^2_{Y|X} - \beta_2^2 \textrm{Var}(W | X)$
#### Precision for Difference of Proportions
When analyzing proportions (means), the mean variance relationship is
critical
Precision is greatest when proportion is close to 0 or 1
Greater homogeneity of groups makes results more deterministic (this is
the goal, at least)
Independent binary outcomes
$\textrm{ind } Y_{ij} \sim B(1, p_j), i = 1, \ldots, n_j; j = 1, 2$
$n = n_1 + n_2; r = n_1 / n_2$
$\theta = p_1 - p_2$,
$\hat{\theta} = \hat{p}_1 - \hat{p_2} = \overline{Y}_1 - \overline{Y}_2$
$\sigma^2_j = p_j(1-p_j)$
$V = (r+1)(\frac{\sigma_1^2}{r} + \sigma_2^2)$
$se(\hat{\theta}) = \sqrt{\frac{V}{n}} = \sqrt{\frac{\sigma^2_1}{n_1} + \frac{\sigma^2_2}{n_2}}$
#### Precision for Odds
When analyzing odds (a nonlinear function of the mean), adjusting for
precision variables results in more extreme estimates
$\textrm{Odds} = \frac{p}{1-p}$
Odds using average of stratum specific $p$ is not the average of stratum
specific odds
Example: Stroke by smoking (in CVD strata)
No association between smoking and CVD in the sample: 10% smokers in
each group
CVD is not a confounder, but is clearly a precision variable
Note that the unadjusted odds ratio is attenuated toward the null
compared to the adjusted odds ratios
-------- ------- ------ ------ ------ ------ ------ ------- ------- -------
$N$ $p$ odds $N$ $p$ odds $N$ $p$ odds
Smoke 1000 0.04 0.04 100 0.50 1.00 1100 0.082 0.089
Nonsmk 10000 0.02 0.02 1000 0.33 0.50 11000 0.048 0.051
Ratio 2.00 2.00 1.75
-------- ------- ------ ------ ------ ------ ------ ------- ------- -------
Diagnosing Confounding
----------------------
### Adjustment for Covariates
We include predictors in an analysis for a number of reasons. In order
of importance\…
1. Scientific question
2. Predictor of Interest
3. Effect Modifiers
4. Adjust for confounding
5. Gain precision
Adjustment for covariates changes the question being answered by the
statistical analysis
Adjustments can be made to isolate associations that are of particular
interest
When consulting with a scientist, it is often difficult to decide
whether the interest in an additional covariate is due to confounding,
effect modification, or precision
The distinction is important because I tend to treat these variable
differently in the analysis
Often the scientific question dictates inclusion of particular
predictors
Predictor of interest: The scientific parameter of interest can be
modeled by multiple predictors (e.g. dummy variables, polynomials,
splines)
Effect Modifiers: The scientific question relates to the detection of
effect modification
Confounders: The scientific question may be state in terms of adjusting
for known (or suspected) confounders
### Confounder Detection
Unanticipated confounding
Some times we must explore our data to assess whether our results were
confounded by some variable
Goal is to assess the “independent effect” of the predictor of interest
on the outcome
Confounders
Variables (causally) predictive of the outcome, but not in the causal
pathway
Best method: Think about the scientific problem beforehand (perhaps draw
DAG)
Using data, often assessed in the control group
Variables associated with the predictor of interest in the sample
Note that statistical significance is not relevant because this tells us
about associations in the population
Detection of confounding ultimately must rely on our best knowledge
about the possible scientific mechanisms
Effect of confounding: A confounder can make the association between the
predictor of interest and the response variable look\…
Stronger than the true association
Weaker than the true association
The complete reverse of the true association (“qualitative confounding”)
Graphical Methods for Visualizing Effect Modification, Confounding, and Precision
---------------------------------------------------------------------------------
Conduct stratified analysis to distinguish between
Effect modifiers
Confounders
Precision variables
### Effect Modifiers
Estimates of treatment effect differ among strata
When analyzing difference of means of continuous data, stratified smooth
curves of the data are non-parallel
Graphical techniques difficult in other settings
### Confounders
Estimates of treatment effect the same across strata, AND
Confounder is causally associated with the response, AND
Confounder associated with the POI in the sample
When analyzing difference of means of continuous data
Stratified smooth curve of data are parallel
Distribution of POI differs across strata
Unadjusted and adjusted analyses give different estimates
### Precision Variables
Estimates of treatment effect the same across strata, AND
Variable is causally associated with the response, AND
Variable is not associated with the POI in the sample
When analyzing difference of means of continuous data
Stratified smooth curve of data are parallel
Distribution of POI same across strata
Unadjusted and adjusted analyses give similar estimates but with smaller
standard errors
